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This report describee e special study of different methods for presentation 
of programmed diagnostic tests. 

The study was carried out on an electronics course with iutegrated feed* 
back during the spring of 1969. 

Two methods for presentation of programed diagnostic tests were compared. 

One method used five-screen tapes lide and the other used TV in the form 
of videotape recording. 

The study comprised ten diagnostic tests in the course* five for each method. 
Evaluation was made on the basis of test points and attitude tests. 

There was no significant difference in the measuring capacity of the two 
methods* but the students" attitudes were in fuvour of the tapeslide form 
of presentation. 
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INTRODUCTION 



Today there is an increasing demand for education. With a limited number 
of teachers there is a need for more rational teaching methods. 

Centrally produced course materiaL with a systematic use of audiovisual 
aids and with built-in diagnostic tests is one way of relieving the teacher 
of much routine work. 

At the Royal Institute cf Technology courses of this kind in electronics 
have been developed by the PE Group. Special efforts have been devoted to 
the development of th* feedback technique necessary for process control 
(report PE-8). 



1 . 1 Course in Transistor Pulse Circuits 

In the autumn of 1967 an electronics course with 14 uniformly built-up 
weekly packages was developed in accordance with the package model in 
figure 1, Each package started with a motivating TV programme (TV) followed 
by independent studies (IS) in the textbook. This was checked with a 
diagnostic test (DT) in class followed by a discussion (D) . Each package 
then concluded with calculation exerciseB ’C) and laboratory experiments (LAB). 




Fig, 1 Package modal for the use of different activities in the 
electronics course. 



The television programmes were produced in cooperation with TRU (The 
Commission, for Television and Radio in Education) which also supported 
tba development of toe diagnostic tests. 

The course was run for the first time in the spring of 1968 as an axperi- 
ment in the systematic use of TV programmes in the teaching of tlect.oqlcs. 
The results of the course were the seme although the ordinary lecturing 
time was cut by 75 % and the remaining 25 X was uiad for TV programmes 
and diagnostic tasts with discussions (report FE-4). 
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1.2 Diagnostic tests 

The diagnostic test as shown in figure 1 was introduced in the model for 
several reacons: 

1. To get a continuous follow-up of students' results for evaluation of 
the teaching model. 

2. To give the teacher an immediate feedback on weak points, thus providing 
him with a better basis for adaptation of the teaching to students' need*. 

3. Individual feedback to students. 

To make this feedback quick, the test used the multiple-choice technique 
and five-screen slide projection. Each question was presented on one screen 
and the four alternatives on the others. The running of the test was program- 
med from tape to make the test situation reproducible. 

The use of multiple choice questions made it possible to correct the answer 
cheets during a short break and then to comment on the result, especially 
on the weak points. The test led to very active discussions on the subject 
and was very much appreciated by the students (report PE-3). 

Later in 1968 an electronic responder system, ESAU, was developed. This 
facilitated the running of the test and permitted direct comments on each 
question and on the answers (report PE-7). 
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2. PROBLEM 

2* 1 Different med ia for test presentation 

The reason for the use of five screens for test presentation was that, 
combined with an electronic responder 3ystem» the distractors could be 
removed at the end of the question. This would reinforce the right answer 
and avoid learning the wrong answers. 

The rather large projection area also facilitated reading of the question 
as well as the alternative answers* 

A five screen slide projection systeta is* of course* neither easy to handle 
nor cheap to install, and this restricted the technique. 

Many other forms of presentation are, of course, possible, e.g. stencil, 
tapes lide with one projector, overhead, TV (live or taped)* 

2.2 TV versus tapes lide 

An interesting alternative is teats recorded on videotape* The TV presen- 
tation of the test is both reproducible and easy to distribute over CCTV 
(Closed Circuit Television) network* Teaching at the new Institute of 
Technology in Linktfping is to a large extent based on CCTV. On courses 
with more than 270 students (the largest lecture nail has 270 seats) there 
will be TV lectures in smaller groups. For this kind of teaching TV is the 
natural choice for presentation if programmed diagnostic tests are to be 
used. 

The main differences between the two forms of presentation are; 

1* Five screen tapeslide has a much larger projection area than TV* 

2. Quastfon and alternative answers can be shown at tha same time with 
slides, which is not always the csae with TV. The small TV scraan 
restricts tha amount of information presantad at the same time* 

Tkeee restrictions in TV presentation are liab’i to affect the measuring 
capacity of the tast and also studant altitudes to the diagnostic tests 
in a negative direction. 

For this reason it is of interest to study the question: To what extent 
does TV presentation of diagnostic tests differ from five screen tepeslide 
presentation in cognitiva measurement capacity, and what are student atti- 
tudes to these forms of presentation of tha tasts? 
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The role of the diagnostic test is to assess the students" cognitive level 
on the week's subject, thus to give both student and teachers an idea of 
the points on which greater efforts required* 

In this special study the result used from the test is only the mesn per- 
centage of right answers* The level of this percentage depends on the 
course material, the students" preparation time, the difficulty of the 
questions, how the questions are presented, and the time for answering etc. 

This result from the test is a relevant measure of the effect of the form 
of presentation of the test, if the otler variables can be controlled. 

The way in which an attempt to do this has been made is described below. 

The choice of a suitable course and the experimental design are here of 
main interest. 

Tbs practical details will be dealt under Preparation of diagnostic tests. 

Student attitudes to the tests are, of course, Important, as they influence 
the way the feedback to the students works, and assessment of student 
attitudes is also discussed. 

3.1 Course cad students 

To be able to compare different forms of presentation for diagnostic tests 
the above mentioned transistor pulse circuits course was used. This course 
already had a set of evaluated diagnostic tasts produced for five screen 
tapeslide presentation. To make a compsrincro between the forms of presen- 
tation in the ordinary teaching situation tasts produced on TV (but with 
the seme questions as before) could be used. 

An experiment of this kind was carried out on the transistor pulse circuits 
course during the spring of 1968, The course was being rdn for the second 
yeai and was taken by 186 students in t ^ (second yeer. Electrical Enginee- 
ring line). 

The course was well suited for thl.i study because it had been thoroughly 
evaluated on two runs with altogether about 300 students in the previous 
year (report H5-4). 

The electronic responder system ESAU was rsady for us# on this courss, 
and the tasts were thus progrsnmii with direct comments on etch question. 
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3*2 Deal gn 

As mentioned above, several factors influence the test result. To isolate 
the f on of presentation factor as veil as possible, the following design 
was used * 

From the 14 packages of the course only 10 were used for the study. Numbers 
1, 12, 13 and 14 were changed frou the previous year and therefore not 
included. 

To get the two forms of presentation evenly distributed over the different 
parts of the course, every second packag;e used tapeslide and the others TV 
for test presentation. 



Presentation 

form 


Teat 

No. 




Tapeslide 


2 4 6 8 10 


even 


TV 


3 5 7 9 11 


odd 



Tests with even numbers (called n even H ) were programmed on tapeslide and 
tests with odi numbers (called "odd* 1 ) on TV. 



This design makes it possible for the same students to be measured with 
both presentation forms and anlalysed both individually and in groups. 
The results from E 2 -68 show that the mean result on "odd" and "even" 
tasta are practically the tame (shown in table 1). 

Table l. Kean results from diagnostic teat E 2 ~68 



— 
T«*t Mo. 


23456/89 10 11 


odd 


even 


Mmji X 
right 

«niw«rt 


55 56 76 72 41 51 72 56 51 51 


57.2 


59.0 



The design also makes it possible for the aama students to compare the 
methods in attitude teste. 



O 

ERIC 



9 



6 . 



3.3 Preparation of diagnostic tests 

To be able to compare the different tests, the same questions were used 
as in the previous year. 

Some problems arose, as the questions sometimes contained too much informa- 
tion to be displayed on the TV screen at the same time. 

Thus the TV presentation of the test required a special technique. The 
question was shown first, and then the alternative answers. This sequence 
was repeated a couple of times. The questions were always put in short 
form at the top of the picture to ensure continuity. 

An example of how one question was transferred from tapeslide form to TV 
is shown in figures 1 and 2. The different sizes of the frames are not 
true to the real test situation, wherj the slide screens are equal and 
much larger than the TV screens. To compensate for this, five TV monitors 
were distributed throughout the lecture hall to obtain a maximum viewing 
distance of 6 m. 

Extra time for answering (5 s) is given in the TV form because of the 
alternation of information. 
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Question: 




Question 7 

Here are two logic modules of the 
same kind with the outputs connected 
in parallel. 

What is the logical output function 
X if we use positive logic? 



Question 20 8 
Answer 20 s 
Question 15 s 
Answer 10 a 



(time for 
answering) 



Alternat ives: 



j What is the logical output function 
' X if we use positive logic? 



a. ABC + DEf 



b. ABC • DEF 



c. (MtC) + tthCtf) 



d. (MM) • (0+C+f) 



Ding 

3 c 

Don*. (end of time for 

answering) 

Comments 

The right answer is a. The modules 
are NAND circuits because only if 
all of B and C are high will 
the transistor saturate and make 
the output low. 

If any of the output transistors 
in parallel is saturated, the 
output X will ba low. The output 
X is thus the inverted sum of 
two AND functions. 



Fift. 2 The same question as in figure l programmed for TV (manuscript form). 
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3.4 Attitude tests 



There are two important criteria of how a diagnostic test works. One is 
how it measures and the other is how it is accepted by the students. 

The first part can be dealt with by analysing the test results. The 
other can be assessed by attitude tests. For this purpose two attitude 
tests were run, the first (I February-69) after package 5 and the second 
(II Hay-69) at the end of the course. These tests ate reproduced in 
report PE-8 together with the distribution of answers. Some of the ques- 
tions will be given later in this report. 

The attitudes will also affect attendance, as there le no compulsion to 
attend lectures. Poor programmes will decrease attendance, as the students 
at the Institute are rational beings and try to optimize their efforts to 
fulfil their own objective^. 
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4. RESUL TS 

4 . 1 Pi agnostic teat results E 2 -69 

In table 2 below the mean results from the diagnostic tests run both for 
and E 2 ~69 are given * 

The percentage of right answers is calculated on the total number of 
students attending the test. 



Table 2. Diagnostic test results from E 2 ~68 and ^2*69 



Test No 




2 


3^ 


4 


5 


6 


7 


8 


9 


10 


11 


Mean X 

right 

an&er 


E 2 -69 


66 


47 


58 


67 


57 


60 


67 


55 


56 


68 


E 2 -68 


55 


56 


76 


72 


41 


51 


72 


56 


51 


51 



As a f'rst check the mean results from -68 and -69 are compared in 
table 3. It will be seen that the overall results on even and odd tests 
are the same for the two years. 




Table 3. Mean results on even and odd diagnostic tests 



Tests 


even - tapesllde 


odd - TV 


Mean X 
right 

answers 


E 2 -69 


60.8 


59.4 


E 2 -68 


59.0 


57.2 



Another factor of interest to compare is how the attendance at even and 
odd tests is distributed in the two courses. This is shown in table 4 and 
figure 3. 



Table 4. Attendance at diagnostic tests E 2 -68 and E 2 -69 



Test No. 






l_. 


A 


5 


6 


7 


_JL 


_JU. 


.. IQ... 


_ u 


AttMtd.DC. 

AS 


E 2 -49 


55 


65 


55 


47 


49 


45 


40 


30 


39 


40 


m 


b 2 -m 


75 


72 


62 


57 


51 


42 


56 


43 


39 


34 



14 



11 



I 



100 





-i— — ■ i ■ ■■ r i ■ "‘v i — i ■ i ■ 1 i i 

2 J * 5 6 7 6 9 10 11 OT Bo. 
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not coeval aory at tha loyal Inatituta of Technology. 
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In table 5 the levels of attendance for even and odd testa are given. 
Table 5, Mean attendance at even 



and odd diagnostic tests 



Tests 




even 


Odd 


Attendance 

Z 


B 2 -69 


47,6 


45.4 


E 2 -68 


56,6 


49.6 



4*2 Statistical analysi s 

For the direct comparison of the two methods of presentation only those 
students who have attended all ten tests will le taken into account. In 
this way the expected means of even and odd tests are the same. 

The results from even and odd are correlated because the same individualii 
are involved. This effect can be eliminated in the calculations by using 
the difference between even and odd test results for each individual. 

Fourteen students attended all ten tests and their mean results on "even 11 

2 

and "odd" are given in table 6, where the difference E - 0 ■ D and D are 
given* 







Table 6. Mean test results for students 
who attended all ten tests for 



Teat 


results 


Difference 




Odd 


Even 


o 

t 

CO 

fl 


D 2 


S.8 


6.0 


-0.2 


0.04 


5.2 


4.2 


1.0 


1.00 


4.0 


4.0 


0.0 


0.00 


4.2 


4.0 


0.2 


0.04 


3.6 


5.4 


-1.8 


3.24 


3.6 


3.4 


-1.8 


3.24 


4.8 


4.4 


0.4 


0.16 


5.2 


4.4 


0.8 


0.64 


4.6 


4.4 


0.2 


0.04 


6.0 


5.8 


0.2 


0.04 


3.8 


4.0 


-0.2 


0.04 


4.4 


3.6 


0.8 


0.64 


5.8 


6.4 


-0.6 


0.36 


4.2 


5.4 


-1.2 


1.44 




E -2.2 
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1 13. 

D * - T 7 - ■ - 0.31 ( fi -4.4 X difference in mean percentage of right answers) 
j 14 

\ This mean deviation can be tested for significance by computing the corres- 

l ponding t value (forcnla 11.9 in Ferguson): 



t - — ■ 1 1 

|/[ne? 



ED 

- (ED) 2 ] /(N-l) 




^2j2 

/[l4 • 10.92 - (-2.2> 2 j/(U-l) 



- -0.65 



With N-l-13 degrees of freedon the t value needed for significance at 5 X 
level in e two tailed test is 2.16 which is not reached in this case. 

Thus there is no support in this study for the assumption that the two 
forms of presentation differ significantly in cognitive measurement capacity. 
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DISCUSSION 



Two different forms of presentation of programmed diagnostic tests have 
been compared . 

The comparison did not show any significant differences in mean test scores 
either for the total group or for the students attending all tests. The 
effects studied were rather comprehensive. The differences between the 
forms of presentation may be referred to several factors: media (slide - TV), 
readability, presentation sequence, attitudes to the media itself. 

As the study was carried cut in the regular teaching at the Royal Institute 
of Technology, no real experiment was possible In which these factors could 
be Isolated. 

Studying the decreasing attendance frequency for the two courses, the ten- 
dency Is the same but the mean level somewhat lower for 82*69. This can 
partly be accounted for by the negative attitude to TV presentation of the 
diagnostic tests, aa shown in attitude questions 1:7 and 11:12. 

The fact that tie attendance at odd tests is somewhat lower than at the 
even tests for both courses can be explained by the decreasing attendance 
towards the end of the term, as the odd tests are one test behind the even 
ones. 

Student attitudes to the course In general have been positive all through 
(questions 1:1 and 11:1), which makes It easier to evaluate the negative 
attitudes. 

An Important factor is how the tests have Influenced the motivation for 
independent studies (question 1:2 and 11:11). The slightly decreasing 
positive attitude In this report can be referred partly to the negative 
attitude* to TV presentation of the testa and partly to the fact that Inde- 
pendent study has Increasingly become the ordinary method of studying 
(report PE-8) . 

On the direct question about the two forms of presentation, the students 
think that slides are better than TV. This la accentuated at the end of 
the course (question 1:7 and 11:12). This can be compared with the diffe- 
rence between colour and black/whlte TV In teaching. The cognitive effects 
are often found to bee the same but attitudes to colour TV are more favour- 






; 
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Appendix 



PRIMARY DATA PROM B ^ -69 
Number of right answers < 7 

Diagnostic test No. Absent =^-i Mean. Attendance. 

I 1 I 1 



1 


2 


3 


4 


5 


6 


7 


8 


9 


10 


11 


13 


ODD 


EVEN 


ODD 


EVEN 


2 




6 


5 


5 


7 


5 


7 


7 


4 


7 


6 


6,00 


5,75 


5 


4 


-1 


-1 


-1 


-1 


-1 


-1 


-l 


-1 


*1 


-] 


-1 


-1 


0,00 


0,00 


0 


0 


-1 


3 


3 


1 


6 


4 


5 


-1 


5 


4 


* . 


2 


5,00 


3,00 


5 


4 


6 


4 


2 


-1 


4 


-l 


-l 


»i 


3 


4 




-1 


3,25 


4,00 


4 


2 


n 5 


5 


2 


3 


4 


7 


-1 


-1 


-l 


(, 


t 


-l 


3,00 


5,25 


2 


4 




2 


-l 


-1 


-1 


-1 


-i 


-1 


-1 


-1 


-1 


-1 


0,00 


2,00 


0 


l 


-1 


8 


6 


5 


6 


-1 


-i 


-1 


-1 


-1 


-1 


-1 


6,00 


6,50 


2 


2 


-1 


-l 


-1 


-1 


-1 


-1 


-l 


-1 


-1 


-) 


-1 


-1 


0,00 


0,00 


0 


0 


0 5 


7 


4 


5 


6 


5 


7 


7 


6 


<) 


6 


7 


5,80 


6,00 


5 


5 


6 


4 


'N 


4 


3 


-1 


-1 


4 


3 


-l 


-1 


-1 


3,00 


4,00 


3 


3 


4 


4 


5 


2 


-1 


-l 


-1 


3 


0 


4 


-1 


-1 


2,50 


3,25 


2 


4 


-i 


-l 


5 


5 


4 


-l 




-l 


3 


-X 


-1 


-1 


4,00 


5,00 


3 


1 


2 


2 


4 


i 


-1 


-1 


-1 


-l 


-1 


-1 


-l 


-1 


4,00 


1,50 


1 


2 


6 


5 


4 


4 


5 


4 


3 


4 


-l 


3 


-1 


4 


4,00 


4,00 


3 


5 


7 


5 


3 


4 


7 


-1 


4 


4 


-l 




7 


4 


5,25 


4,33 


4 


3 


5 


7 


4 


4 


5 


-! 


6 


6 


-1 


-1 


7 


5 


5,50 


5,67 


4 


3 


-1 


3 


2 


7 


-1 


4 


4 


3 


5 


5 


5 


3 


4,00 


4,40 


4 


5 


-1 


-1 


-1 


-1 


-1 


-1 


-1 


-1 


-1 


-1 


-1 


-l 


0,00 


0,00 


0 


0 


4 


6 


6 


5 


6 


-1 


-1 


-1 


D 


-1 


-1 


-l 


4,00 


5,50 


3 


2 


O-i 


-1 


-1 


-1 




3 


-1 


-1 


-1 


-1 


-1 


-1 


0,00 


3,00 


0 


1 


5 


-1 


1 


4 


6 


6 


1 




-1 


-1 


-t 


-1 


2,b7 


5,00 


3 


2 


5 


7 


2 


4 


4 


5 


2 


-1 


-1 


-1 


-1 


-1 


2,67 


5,33 


3 


3 


0 4 


5 


5 


2 


7 


4 


4 


6 


4 


4 


6 


5 


5,20 


4,20 


5 


5 


-1 


-1 


0 


5 


3 


-l 


-1 


-1 


-1 


-1 


-1 


-1 


1,50 


5,00 


2 


1 


-1 


-1 


3 


-1 


-1 




6 


-1 


-1 


4 


7 


6 


5,33 


5,50 


3 


2 


4 


5 


3 


5 


5 




3 


5 


-1 


3 


4 


6 


4,25 


4,40 


4 


5 


2 


-1 


1 


5 


6 




-i 


«i 1 


-1 


-l 


•l 


-1 


3,50 


4,00 
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